Nonlinear Markov Games ∗

I am going to put forward a program of the analysis of a new class of stochastic games that I call nonlinear Markov games, as they arise as a (competitive) controlled version of nonlinear Markov processes (an emerging field of intensive research, see e.g. [3], [4],[5]). This class of games can model a variety of situation for economics and epidemics, statistical physics, and pursuit evasion processes. The discussion below is mostly taken from the author’s monograph in preparation [1]. I shall start by introducing the (yet not very well known) concept of nonlinear Markov chains. 1 Nonlinear Markov chains A discrete in time and space nonlinear Markov semigroup Φ, k ∈ N, is specified by an arbitrary continuous mapping Φ : Σn → Σn, where the simplex Σn = {μ = (μ1, ..., μn) ∈ R+ : n ∑ i=1 μi = 1} represents of course the set of probability laws on the finite state space {1, ..., .n}. For a measure μ ∈ Σn the family μ = Φμ can be considered as an evolution of measures on {1, ..., .n}. But it does not yet define a random process (finite-dimensional distributions are not specified). In order to get a process one has to choose a stochastic representation for Φ, i.e. to write it down in the form Φ(μ) = {Φj(μ)}j=1 = { n ∑ i=1 Pij(μ)μi}i=1, (1.1) where Pij(μ) is a family of stochastic matrices ( ∑d j=1 Pij(μ) = 1 for all i), depending on μ (nonlinearity!), whose elements specify the nonlinear transition probabilities. For any given Φ : Σn 7→ Σn a representation (1.1) exists, but is not unique. For instance, there exists a unique representation (1.1) with an additional condition that all matrices Pij(μ) = P̃ij(μ) are one dimensional: P̃ij(μ) = Φj(μ), i, j = 1, ..., n. (1.2) ∗Talk given at Adversarial and Stochastic Elements in Autonomous Systems Workshop, 22-24 March 2009, at FDIC, 3501 Fairfax Drive, Arlington, VA 22226. †Department of Statistics, University of Warwick, Coventry CV4 7AL, UK. Email: v.kolokoltsov@warwick.ac.uk